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1.0  INTRODUCTION 

This  report  describes  information  gathered  about  institutes  in  the  UK  undertaking  research  into 
Augmented,  Mixed  and  Virtual  Environments  (AMVE)  for  military  applications  of  Virtual  Reality.  It  also 
includes  information  on  activities  being  undertaken  in  the  civil  sector  that  might  be  relevant  to  the 
military. 

The  information  is  described  under  three  heading:  UK  Defence  (Dstl/QinctiQ  work),  UK  Academic 
Institutions,  and  UK  Commercial  Companies. 


2.0  UK  DEFENCE  -  SOME  CURRENT  PROGRAMMES 

2.1  UK  Mission  Training  via  Distributed  Simulation 

Under  the  sponsorship  of  MoD  a  programme  of  applied  research  has  been  undertaken  to  explore  the 
benefits  to  be  gained  from  using  networks  of  simulator,  or  Virtual  Training  Environments  (VTEs), 
for  aircrew  collective  mission  training1.  Use  of  networked  simulation  in  this  context  has  become  known  as 
Mission  Training  through  Distributed  Simulation  (MTDS). 

The  original  research  remit  was  to  investigate  the  technical  issues  associated  with  linking  legacy  and  new 
generation  military  training  simulators,  to  assess  the  potential  for  conducting  aircrew  collective  training. 
The  primary  hypothesis  was  that  training  simulators  could  be  connected  together  to  provide  a  common 
synthetic  battlespace  capable  of  supporting  the  collective  training  needs  of  front-line  combat  ready 
aircrew. 

MoD’s  research  needs  were  founded  in  a  growing  interest  in  the  potential  of  VE  for  training  purposes. 
The  big  assumption  was  that  by  transferring  certain  tasks  from  the  live  training  environment  into  the  STE, 
the  same  level  of  operational  effectiveness  could  be  maintained  but  at  less  cost.  Today  this  premise  is  as 
hotly  debated  as  in  the  early  1990’s.  A  balance  between  live  and  synthetic  training  has  still  to  be 
determined.  This  is  not  surprising,  as  an  optimal  balance  may  not  exist  across  the  complete  spectrum  of 
operational  training.  It  is  a  complex  issue  and  the  debate  is  likely  to  continue  for  some  time  yet.  However 
some  notional  indication  of  the  balance  is  undoubtedly  a  requirement  when  procuring  any  military 
equipment  that  requires  a  synthetic  training  component. 

Despite  the  growing  interest  in  STEs  and  the  perceived  potential  for  mission  training  via  distributed 
simulation,  a  proven  SE  capable  of  providing  collective  training  for  front-line  aircrew  did  not  exist.  Thus 
whilst  this  research  activity  had  been  structured  in  such  a  way  as  to  enable  some  very  focused  technical 
studies  to  take  place,  it  became  increasingly  obvious  that  the  general  hypothesis  must  be  tested  via 
empirical  research  methods.  The  research  team  therefore  designed  a  series  of  practical  experiments  (trials), 
conducted  under  the  banner  heading  of  RAPTORS  (Research  into  Aircrew  Performance  and  Training 


1  ‘Collective  mission  training’  is  defined  as  two  or  more  teams  training  to  interoperate  in  an  environment  defined  by  a  common 
set  of  collective  mission  training  objectives,  where  each  team  fulfils  a  different  military  role.  NATO  SAS-013  Study. 


RTO-TR-HFM-1 21  -Part-1 


6-1 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

01  DEC  2005 

2.  REPORT  TYPE 

N/A 

3.  DATES  COVERED 

4.  TITLE  AND  SUBTITLE 

Institutes  in  the  UK  Undertaking  Research  into  Augmented,  Mixed  and 
Virtual  Environments  (AMVE)  for  Military  Applications  of  Virtual 

Reality 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROIECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

DSTU 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

1 1 .  SPONSOR/MONITOR' S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release,  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

See  also  ADM001858,  Virtual  Environments  for  Intuitive  Human-System  Interaction  (Environnements 
virtuels  dinteraction  Homme-Systeme  Intuitive).,  The  original  document  contains  color  images. 

14.  ABSTRACT 

15.  SUBIECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 
ABSTRACT 

uu 

18.  NUMBER 

OF  PAGES 

22 

19a.  NAME  OF 
RESPONSIBLE  PERSON 

a.  REPORT 

unclassified 

b.  ABSTRACT 

unclassified 

c.  THIS  PAGE 

unclassified 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


INSTITUTES  IN  THE  UK  UNDERTAKING  RESEARCH  INTO  AUGMENTED,  MIXED  AND 
VIRTUAL  ENVIRONMENTS  (AMVE)  FOR  MILITARY  APPLICATIONS  OF  VIRTUAL  REALITY 


ORGANIZATION 


using  Operationally  Realistic  Scenarios).  To  ensure  that  data  obtained  from  these  experiments  was 
assessing  genuine  military  training  potential,  the  experimental  subjects  comprised  front-line  aircrew. 
A  bespoke  synthetic  COMAO  test-bed2  was  developed  specifically  to  support  the  RAPTORS  series  of 
trials  and  comprised: 

•  A  large  package  of  aircraft  (~  40)  comprising 

•  Eight  manned  simulators  flown  by  front-line  aircrew 

•  A  manned  E3-D  fighter-controller  station 

•  Other  friendly  platforms  represented  by  role  players  &  computer  generated  forces  (CGF) 

•  Hostile  forces  represented  by  two  man-in-the-loop  simulators,  CGF  &  role  players 

•  Complex  tactical  scenario,  closely  based  on  a  real-world  theatre 

•  Geo-specific  location,  targets,  procedures,  etc. 

•  Dynamically  controllable  threat  environment 
•  e.g.  In-mission  re-tasking  of  CGF 

•  War  operations  room  and  Exercise  Management  suite 

•  Planning,  briefing  and  debriefing  complex 

The  test-bed  provided  a  comprehensive  trainee-centric  STE  created  over  a  secure  simulation  network  that 
supported  exercise,  technical  and  tactical  management  of  the  scenario.  The  complexity  of  the  RAPTORS 
COMAO  environment  is  depicted  below. 
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Schematic  of  the  RAPTORS  COMAO  Environment. 


2 

The  test-bed  was  developed  by  DERA  at  the  Bedford  site.  It  is  now  a  QinetiQ  facility,  although  a  limited  number  of 
components  are  customer  owned. 
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To  date  the  RAPTORS  series  of  trials  has  comprised  of  Ebb  and  Flow  (Feb  2000),  SyCOE  (Jan  2001), 
VirtEgo  (Nov  2001)  and  SyCLONE  (Feb  2003).  Each  trial  has  looked  at  different  aspects  of  collective 
training  from  a  technology  and  human  factors  perspective3.  The  current  UK  MTDS  programme  was 
initiated  due  to  the  success  of  these  trials. 

2.2  War  Fighting  Experiment  (WFE)/Synthetic  Environment  (SE)  Demonstrator  and 
Training  Strategy 

The  aim  of  this  work  is  to  create  and  demonstrate  the  tools  and  techniques  to  capture  capability  gaps  and 
evaluate  the  effectiveness  of  Indirect  Battlefield  Engagement  (IBE)  equipment  in  realistic  joint 
experimental  settings.  The  core  of  the  first  year’s  work  comprises  two  elements.  The  first  of  these  is  a 
training  strategy  study,  which  will  address  the  overall  collective  training  required  across  IBE.  The  second 
is  the  development  of  integrated  live/virtual  simulation  techniques  (“augmented  reality”)  enabling  a 
number  of  simulated  attack  helicopters  to  take  part  in  a  live  exercise,  for  force  evaluation  and 
experimentation  purposes.  The  second  year  is  focused  on  developing  an  enduring  live/virtual  capability, 
by  linking  Synthetic  Environments  (SEs)  with  Area  Weapons  Effects  Simulator  (AWES),  and  broadening 
the  extent  of  the  IBE  SE  representation  by  bringing  in  a  future  ground-based  indirect  fire  element. 

2.3  Human  Effectiveness  (C2) 

It  is  recognised  that  the  challenge  over  the  next  decade  will  be  to  assist  Intelligence,  Surveillance,  Target 
Acquisition  and  Reconnaissance  (ISTAR)  mission  specialists  in  exploiting  the  total  range  of  data  that  will 
be  made  available  to  them  whilst  at  the  same  time  preventing  them  from  becoming  swamped  with 
unmanageable  information.  The  MoD  customer  requires  a  focus  for  the  development  of  human  factors 
policy  for  the  use  and  exploitation  of  ISTAR  assets,  including  the  capture  and  promulgation  of  best 
practices  on  Human  Computer  Interaction  (HCI)  interface  design  and  workload  reduction.  A  primary 
objective  is  to  determine  the  benefits  of  using:  multimedia  HCIs  in  Intelligence  Databases  and  Image 
Product  Libraries;  touch  screens  and  pressure  sensitive  pens  in  the  exploitation  task;  speech  recognition 
technology  in  populating  intelligence  reports  and  retrieving  intelligence  information;  fusing  intelligence 
products  to  increase  the  effectiveness  of  the  exploitation  task. 

2.4  Improving  the  Representation  of  Human  Variability  in  CGF 

The  aim  of  this  research  activity  is  to  enhance  the  representation  of  Human  Variability  in  Computer 
Generated  Forces  (CGF),  taking  factors  such  as  stress,  fatigue,  cold,  heat,  night  or  day  operations  etc  into 
account;  thus  making  SEs  more  credible  for  users.  The  outcome  of  this  work  will  be  UK  MOD’s  principal 
contribution  to  the  development  of  the  US  OneSAF. 

2.5  Realistic  Synthetic  Environments  for  Secure  Command  Information  Service 

This  work  aims  to  determine  and  demonstrate  how  an  SE  can  be  constructed  to  provide  an  intuitive 
environment  for  users  to  easily  specify,  develop  and  exercise,  through  the  stimulation  of  a  secure 
command  system. 

2.6  Automatic  Extraction  of  3D  Information  from  Geospatial  Data 

Building  on  previous  work,  this  study  aims  to  automate,  and  reduce  the  time  taken  to  extract  (satellite) 
imagery  data  and  create  a  new  rapid  and  cost  effective  Synthetic  Natural  Environment  (SNE)  terrain 


3  The  UK  MTDS  Capability  Working  Group  was  instigated  after  Trial  VirtEgo.  Trials  Ebb  and  Flow  and  SyCOE  were  funded 
solely  by  the  ARP.  VirtEgo  and  SyCLONE  were  part  funded  by  Eleadquarters  Strike  Command  (E1QSTC)  and  were  designed 
to  support  both  research  and  training  objectives. 


RTO-TR-HFM-1 21 -Part-1 


6-3 


INSTITUTES  IN  THE  UK  UNDERTAKING  RESEARCH  INTO  AUGMENTED,  MIXED  AND 
VIRTUAL  ENVIRONMENTS  (AMVE)  FOR  MILITARY  APPLICATIONS  OF  VIRTUAL  REALITY 


ORGANIZATION 


database  generation,  containing  representations  of  true  3D  features  such  as  buildings  and  mountains. 
The  work  will  open  up  the  potential  for  using  better  SE  for  Mission  Rehearsal. 

2.7  Dismounted  Infantry  Virtual  Environment  (DIVE) 

This  work  set  out  to  exploit  current  and  future  modified  games  technology  to  develop  a  low  cost, 
networked  PC  based  system  for  supporting  dismounted  urban  operations  operational  analysis.  AS  part  of 
this  research  programme  a  training  effectiveness  trial  was  undertaken.  The  virtual  environment  was 
provided  using  a  modification  of  the  commercial  PC-game  HalfLife”.  The  aim  of  this  trial  was  to 
investigate  the  utility  of  the  DIVE  environment  as  a  training  tool  using  an  example  of  Urban  Operations 
Training  at  Team  and  Section  level. 

This  was  assessed  using  a  mix  of  subjective  (questionnaire)  self-assessment  and  Subject  Matter  Expert 
(SME)  observation  of  comparable  tasks  executed  in  the  live  and  virtual  domain.  Training  transfer  effect 
was  observed  in  two  groups  of  two  teams  who  followed  different  progressions  of  live  and  DIVE  training. 
A  second  comparison  was  made  at  Section  level. 

Overall,  the  trial  achieved  its  aims  in  demonstrating  that  Synthetic  Environments  (SE)  such  as  DIVE  have 
a  role  to  play  in  training  some  Dismounted  Infantry  roles.  The  following  key  results  are  of  note: 

•  Additional  evidence  of  close  correlation  of  team  performance,  mission  timings  and  outcomes  in 
live  and  DIVE  environments. 

•  No  significant  performance  differences  between  teams  following  different  DIVE  and  live  training 
progressions  at  fireteam  level. 

•  Indications  of  benefit  of  using  DIVE  to  train  at  Section  level. 

•  Strong  subjective  support  from  trial  participants  for  the  use  of  DIVE-type  systems  low-level 
command  training  applications  and  the  benefit  of  playback-type  After  Action  Reviews  (AAR) 
enabled  by  DIVE. 

2.8  Using  SE  in  Support  of  Department  of  State  Activities 

This  work  focuses  on  the  (short  and  long-term)  application  of  SE  tools  and  techniques  to  the  MOD 
Department  of  State  function  in  Military  Assistance  to  Civil  Authorities  (MACA).  The  formulation  and 
execution  of  effective  policy  in  times  of  crisis  is  difficult  due  to  the  complex  interplay  between  political 
and  socio-economic  factors.  The  tools  to  be  developed  and  demonstrated  would  aid  the  formulation  and 
execution  of  crisis  management  policy  and  offer  a  SE  training  tool  for  those  involved  in  such  crisis 
management. 

The  aims  of  this  research  are  twofold: 

•  To  identify  areas  where  existing  SE  tools  can  be  used  for  modelling  MACA  activities,  identify 
where  there  is  a  gap  in  SE  capability,  or  where  SE  technology  is  immature  for  MACA 
applications; 

•  Where  SEs  have  utility,  this  programme  will  aim  to  devise  a  tool  framework  with  appropriate 
interfaces  to  assist  government  departments  in  the  formulation  and  execution  of  policy  and 
planning  in  the  MACA  domain  and  research  the  underpinning  architecture  required  to  implement 
an  appropriate  set  of  training  tools.  The  benefit  of  using  SEs  for  MACA  activities  include  the 
ability  to  explore  alternative  command  structures,  organisations  and  processes,  support  multi¬ 
agency  training  through  distributed  means  with  more  effective  AAR  and  provide  decision  support 
tools  which  can  monitor  crisis  development  as  well  as  assist  in  resource  and  action  planning. 
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2.9  Avatar  Mobile  Instructor 

This  work  is  investigating  the  Military  use  of  Commercial  Avatars  (the  embodiment  of  a  human  form 
within  a  SE)  in  the  context  of  pre-deployment  training.  It  aims  to  determine  whether  Avatars  could 
provide  effective  training  with  the  minimum  demand  for  qualified  manpower  (training  staff).  This  could 
allow  for  increased  currency  of  information  by  taking  training  material  content  from  a  centralised  data 
source.  The  programme  will  culminate  in  a  demonstration  of  an  Avatar  based  system  providing  a 
(re)configurable  system  in  appearance  and  functionality  terms. 


3.0  UK  DEFENCE  RESEARCH  FACILITIES 

3.1  Human  Factors  Integration  Defence  Technology  Centre4 

Aerosystems  International  (Ael)  has  been  awarded  a  contract  by  the  MoD  to  establish  the  Defence 
Technology  Centre  (DTC)  for  Human  Factors  Integration  (HFI).  Ael  will  lead  a  consortium  fonned  from 
leading  industrial  and  academic  groups  which  include  MBDA  Missile  Systems,  Fockheed  Martin  UK  Ftd 
Integrated  Systems,  systems  and  software  house  SEA,  VP  Defence  and  the  Universities  of  Birmingham, 
Brunei  and  Cranfield.  The  consortium  will  use  virtual  enterprise  technology  to  link  consortium  members 
and  a  shared  data  environment  to  allow  all  HFI  DTC  Stakeholders  access  to  the  results  of  the  programme. 

3.2  The  Applied  Research  Technology  Demonstrator  (ARTD) 

The  Applied  Research  Technology  Demonstrator  (ARTD)  has  been  constructed  to  test  and  evaluate  new, 
emerging  technologies  in  an  enclosed,  simulated  battle  environment.  Essentially  it  is  a  ‘plug-in-and-play’ 
unit,  where  armed  forces,  MOD  representatives  and  their  industry  suppliers  can  test  new  technologies  by 
seeing  how  well  they  perform  in  real  scenarios,  played  out  in  a  computer  simulated  environment. 

The  facility  itself  is  a  series  of  rooms  that  can  be  linked  to  one  another,  partitioned  to  create  separate 
working  areas,  or  opened  up  to  create  virtual  battle  spaces.  Units  can  be  added  or  omitted  as  desired. 
State-of-the-art  computers  can  stage  any  given  scenario  and  even  newly  designed  systems  can  be  added  to 
the  simulation  mix. 

As  a  member  of  the  Combined  Federated  Battle  Faboratory  (CFBF),  a  consortium,  developed  between  the 
US,  NATO,  Australia,  Canada,  New  Zealand  and  the  UK,  ARTD  is  now  part  of  a  wide  network  of  similar 
facilities  in  friendly  countries.  It  can  operate  24  hours  a  day  if  necessary,  linking  with  these  other  nations’ 
facilities  over  secure  links,  broadening  the  scope  of  research  now  available  to  the  UK  MOD. 

3.3  Battlespace  Management  Evaluation  Centre  (BMEC)  -  British  Aerospace 

The  joint  MoD/Industry  partnership  known  as  NITEworks  (Network  Integration  Test  and  Experimentation 
works)  will  use  the  BMEC  as  a  hub  for  linking  other  MoD  and  industry  battlelabs  together  to  explore 
network  centric  warfare.  Other  industry  battlelabs  include  those  at  Dstl  (the  ARTD)  and  QinetiQ.  Other 
companies  have  also  been  asked  to  identify  facilities  that  could  potentially  link  to  the  overall  NITEworks 
programme. 


4.0  UK  ACADEMIC  INSTITUTIONS 

Many  UK  Universities  are  involved  with  research  into  Virtual  Reality  and  Augmented  Environments. 
The  main  universities  carrying  out  work  of  interest  are: 

•  Cardiff  University 


4  http://www.hfidtc.com/hfidtc/index.htm 
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•  Hull  University 

•  Loughborough  University 

•  Nottingham  University 

•  Royal  Military  College  of  Science  (Shrivenham  and  Cranfield  University) 

4.1  Cardiff  University 

4.1.1  C-HIVE  -  Cardiff  Human  Interfaces  and  Virtual  Environments  Laboratory5 

Researchers  in  the  C-HIVE  focus  on  two  primary  areas  of  virtual  environments  (VEs;  virtual  reality 
worlds):  (a)  experimental  investigations  of  the  interface  between  people  and  VE  systems  (human- 
computer  interaction),  and  (b)  the  use  of  VEs  to  investigate  other,  real-world  problems.  All  of  the  research 
is  multi-disciplinary. 

4.1.2  Published  Literature 

Ruddle,  R.A.,  Huddart,  S.A.,  and  Jones,  D.M.  (1999).  Interaction  in  Immersive  Virtual  Environments: 
Rotating  Objects  with  an  Instrumented  Prop.  Proceedings  of  the  Human  Factors  and  Ergonomics  Society 
43rd  Annual  Meeting,  pp.  1214-1218. 

Ruddle,  R.A.,  Payne,  S.J.,  and  Jones,  D.M.  (1999).  The  Effects  of  Maps  on  Navigation  and  Search 
Strategies  in  Very-Large-Scale  Virtual  Environments.  Journal  of  Experimental  Psychology:  Applied,  5, 
pp.  54-75. 

Ruddle,  R.A.,  Payne,  S.J.,  and  Jones,  D.M.  (1999).  Navigating  Large-Scale  Virtual  Environments:  What 
Differences  Occur  Between  Helmet-Mounted  and  Desk-Top  Displays?  Presence:  Teleoperators  and  Virtual 
Environments,  8,  pp.  157-168. 

4.2  Hull  University 

4.2.1  Hull  Immersive  Visualization  Environment  (HIVE)6 

HIVE  provides  state-of-the-art  visualization,  interaction  and  computing  technology  and  related  support  for 
both  University  departments  and  industry,  and  is  dedicated  to  helping  researchers  and  industry  make 
effective  use  of  emergent  technologies. 

HIVE  Facilities: 

•  Stereoscopic  vision  (using  immersive  workwall  or  desktop  PC’s) 

•  Virtual  and  augmented  reality 

•  Virtual  prototyping 

•  Collaborative  design  reviews 

•  Development  of  virtual  environment  trainers 

•  Simulation  of  urban  development  and  terrain 

•  Scientific  and  medical  visualization 


5  http://www.cf.ac.uk/psych/ruddle/C-HIVE/ 

6  http  ://www. hul  1 . ac  ,uk/hi ve/index .htm 
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•  Scene  capture  and  reverse  engineering  using  stereo  cameras,  motion  tracking,  CMM 

•  Haptic  interaction  with  visualizations 

•  High  performance  computing 


Computer  Science  Staff  around  the  VisionStation. 


4.3  Loughborough  University 

4.3.1  AVRRC  -  Advanced  VR  Research  Centre7 

The  Advanced  VR  Research  Centre  undertakes  research  in  the  area  of  immersive  and  pervasive  computing 
environments.  Specific  applications  for  the  research  involve  engineering  and  scientific  visualisation. 
The  multi-disciplinary  research  undertaken  by  the  Centre  is  underpinned  by  in-depth  investigations 
involving  the  associated  human  factors  issues.  For  example:  Development  of  intuitive  GRID  working 
environments  and  Improving  remote  visualisation  across  GRID  architectures. 

The  AVRRC  employs  a  range  of  specialised  facilities  including: 

•  Panoramic  Reality  Centre 

*  5M  Vision  Dome®8.  Thus  is  a  fully  immersive  multi-user,  single  projection  Virtual  Reality 
environment  for  interactive  space  development.  Upon  entering  the  VisionDome,  users  are 
completely  drawn  into  a  fully  immersive  180  degree  hemispheric  screen.  The  tilted  screen  is 
positioned  so  as  to  fill  the  users’  field-of-view,  creating  an  incredible  sense  of  immersion.  Users 
experience  vivid  images  that  take  on  depth  via  the  unique  optical  system.  The  VisionDome  is 
further  enhanced  by  an  environmental  sound  system.  Users  of  the  VisionDome  do  not  have  to 
wear  head  mounted  displays,  stereo  glasses,  or  other  restrictive  devices.  The  VisionDome  can  be 
used  for  multi-user,  multi-sensory  display  for  simulation,  training,  design,  engineering,  product 
display,  energy  exploration  &  production,  education,  medical  services  and  entertainment. 
A  second  slightly  smaller  Vision  Dome  is  also  available  and  can  run  in  conjunction  with  the  5M 
Vision  Dome. 


7  http://www.avrrc.lboro.ac.uk/ 

8  http://www.avrrc.lboro.ac.uk/Visiondome_facility.html 
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•  Immersive  Wall 

•  SmartSpace 

•  AccessGrid  -  currently  undergoing  testing 

•  Wearable  Computing  Environments 


Inside  the  VisionDome. 


Loughborough  University  is  also  undertaking  an  extensive  research  programme  which  is  focussing  on 
developing  an  understanding  of  the  complex  human  factors  issues  behind  augmenting  the  real  world  with 
virtual  information.  This  work  links  with  their  research  on  wearable  computing  systems  and  the  University 
has  been  conducting  extensive  research  in  the  field  of  Augmented  Reality  as  a  means  of  improving  the 
cognitive  performance.  Much  of  this  work  is  funded  by  the  UK  Ministry  of  Defence. 
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4.3.2  Recently  Published  Literature 

Stedmon,  W.,  Kalawsky,  R.S.,  Moore,  P.M.,  Aung,  M.,  Purcell,  J.,  Reeh,  C.,  and  York,  T.,  “It’s  not  what 
you  wear,  it ’s  how  you  wear  it:  Human  Factors  of  Wearable  Computers  ”,  Centre  for  Human  Sciences, 
Defence  Evaluation  and  Research  Agency,  UK,  IFC  Research  Group,  Advanced  VR  Research  Centre, 
Loughborough  University,  UK. 

Stedmon,  A.W.,  Kalawsky,  R.S.,  Hill,  K.,  and  Cook,  C.A.,  “Old  Theories,  New  Technologies  Developing 
Guidelines  for  the  Cognitive  Ergonomics  of  Augmented  Reality  ”,  Paper  accepted  for  presentation  at  the 
IEA  2000/HFES  2000  Congress.  July  30th-  August  4th,  2000,  San  Diego,  California. 

JTAP  Project  305,  Human-Computer  aspects  of  Virtual  Design  Environments9 . 


Sony  Glasstron  Augmented  Reality  Head  Mounted  Display. 


4.4  University  of  Nottingham 

4.4.1  Collaborative  Virtual  ENvironments  -  COVEN10 

COVEN  was  a  four-year  project  (1995-1999)  focusing  on  the  development  of  a  computational  service  for 
teleworking  and  virtual  presence.  The  overall  objective  of  the  project  is  to  provide  the  facilities  needed  to 
support  future  cooperative  teleworking  systems.  The  project  is  a  unique  endeavour  that  brings  together 
expertise  on  communications  infrastructures;  computer  supported  cooperative  work;  virtual  reality  (VR) 
and  animation  to  provide  support  for  a  European  wide  distributed  virtual  environment  (VE).  This  virtual 
environment  will  enable  groups  of  geographically  disparate  users  to  work  together  by  inhabiting  common 
information  spaces.  Partners  in  this  work  were: 

•  Arax,  Division  Ltd. 


9 

http://www.avrrc.lboro.ac.uk/JTAP305introd.html 

10  http://www.crg.cs.nott.ac.uk/research/projects/Coven/ 
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•  EPFL  -  Swiss  Federal  Institute  of  Technology 

•  Integrated  Information  Systems  Ftd. 

•  KPN  Research 

•  Fancaster  University 

•  Swedish  Institute  of  Computer  Science 

•  Thomson-CSF  -  Faboratoire  Central  de  Recherches 

•  TNO  Physics  and  Electronics 

•  University  College  Fondon 

•  University  of  Geneva  -  Mira  Fab 

•  University  of  Nottingham 

4.4.2  Structured  Evaluation  of  Training  in  Virtual  Environments  (STRIVE)11 

A  review  of  existing  case-studies  on  VE  training  applications  (VETs)  was  carried  out  to:  examine  the  type 
of  training  applications  and  VR  systems  being  considered;  the  state  of  development  of  these  applications; 
and  results  of  any  evaluation  studies.  From  this  review  it  was  seen  that  the  type  of  VETs  being  explored 
involved  the  training  of  navigation  skills  or  psycho-motor  skills.  However  generally  these  applications  had 
been  ‘proof-of-concepts’  and  little  evaluation  had  been  performed.  Of  the  evaluation  studies  that  existed, 
generally  usability  issues  with  the  systems  had  confounded  the  results.  Therefore  it  was  felt  that  desktop 
VR  would  be  the  most  appropriate  system  to  focus  on,  as  it  has  fewer  of  the  side  effects  associated  with 
other  systems.  Furthermore  desktop  VR  has  been  highlighted  as  the  most  popular  VR  system  because  it 
requires  very  little  initial  investment. 

In  order  to  perform  a  structured  evaluation,  existing  theories  of  training  and  evaluation  were  reviewed. 
It  was  clear  that  evaluation  was  inter-linked  with  the  development  process  of  a  training  application, 
therefore  using  these  theories,  a  framework  for  developing  VETs  was  suggested.  This  framework 
examined  the  stages  of  proposing,  specifying,  building  and  evaluating  VETs.  Then  using  this  framework, 
two  VETs  were  developed  to  be  examined.  There  were  a  number  of  conclusions  that  were  drawn  from 
these  experiments.  In  particular  this  work  highlighted  the  many  areas  in  the  development  process  of  an 
effective  VET  that  still  need  addressing.  There  are  still  tools  to  be  developed  for  evaluating  VETs  and 
much  more  evaluation  studies  required  from  academia  and  industry  to  input  information  back  into  the 
development  process.  Therefore  VEs  are  still  in  their  early  stages  but  it  is  possible  to  recognize  the  gaps  in 
the  development  process  that  need  to  be  addressed  before  implementation. 

4.4.3  Recently  Published  Literature 

Stedmon,  A.W.  and  Stone,  R.J.  Re-viewing  Reality:  Human  Factors  of  Synthetic  Training  Environments, 
International  Journal  of  Human-Computer  Studies  Vol.  55,  No.  4:  675-98,  October  2001. 

4.5  Royal  Military  College  of  Science  at  Cranfield  University 
4.5.1  Flight  Deck  Officer  Training12 

Currently  the  British  Royal  Navy  Flight  Deck  Officers  (FDO)  are  trained  at  RNAS  Culdrose,  Cornwall, 
England.  Although  their  training  is  shore-based,  they  make  extensive  use  of  real  simulation,  learning  to 

1 1  http://www.virart.nottingham.ac.uk/Projects_STRrVE.htm 

12 

http://www.rmcs.cranfield.ac.uk/ssel/train.htm 
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direct  real  helicopters  onto  a  landing  area.  If,  however,  the  weather  conditions  restrict  aircraft  flights  or 
aircraft  are  unavailable,  then  the  training  makes  use  of  a  virtual  simulation  system.  The  current  FDO 
training  simulator  at  RNAS  Culdrose  requires  another  person,  typically  the  instructor,  to  fly  the  helicopter, 
in  response  to  the  signals  given  by  the  trainee. 


The  FDO  Training  Simulator. 

This  RMCS  project  was  an  investigation  into  the  use  of  virtual  reality  to  improve  the  current  simulator. 
It  was  envisaged  that  in  such  a  system  the  trainee  will  be  wearing  a  head  mounted  display  (HMD)  through 
which  he  is  presented  with  the  view  as  seen  on  the  flight  deck  of  a  sea-going  vessel  (e.g.  frigate).  Using 
two  three-dimensional  trackers  the  hand  movements  of  the  trainee  were  monitored  and  computer  software 
determined  the  signal  that  the  user  is  giving.  Once  the  type  of  signal  had  been  determined  it  was  fed 
directly  into  the  dynamics  model  of  the  helicopter,  which  would  then  respond.  Thus  if  the  move  left  signal 
was  given  the  helicopter  would  move  to  the  left.  This  would  remove  the  instructor  from  actually  flying  the 
helicopter  and  allow  him  to  concentrate  on  his  main  task  of  instructing.  In  addition  such  a  system  would 
allow  for  more  than  one  trainee  to  be  trained  at  a  time  and  would  allow  all  of  the  benefits  of  computer- 
based  training  to  be  applied. 

4.5.2  Parachute  Training 

The  Parachute  Training  Simulator  allows  a  trainee  parachutist  to  gain  experience  in  the  handling  of  a 
parachute  while  immersed  in  a  virtual  world.  The  current  application  runs  on  a  PC,  running  Microsoft 
Windows.  It  was  built  using  Superscape’s  VRT  product.  Using  a  head-mounted  display  (HMD), 
the  trainee  is  presented  with  a  view  of  the  world  as  they  descend  from  an  altitude  of  3000ft.  The  use  of  the 
HMD  allows  the  user  to  look  in  any  direction  thus  giving  a  complete  field  of  view  that  would  not  be 
possible,  or  too  costly,  with  a  monitor  or  projection  system.  The  brake  toggles  used  by  a  parachutist  to 
control  their  descent  are  modelled  by  means  of  two  joysticks  that  are  interfaced  to  the  serial  ports  on  a  PC. 
These  inputs  control  the  dynamics  model  of  the  parachute. 
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The  Parachute  Training  Simulator-  1. 


It  is  the  aim  of  the  trainee  to  manoeuvre  themselves,  using  the  brake  toggles  to  traverse  a  course  of  control 
gates  that  an  instructor  has  placed  in  the  virtual  world  and  to  land  at  a  specified  location. 


The  Parachute  Training  Simulator -2. 


The  simulator  allows  for  a  variety  of  malfunctions  in  the  operation  of  the  parachute: 

•  Line  twists 

•  Line  bunched  together 

•  End  cells  closed  (slider  up  or  down) 
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•  Streamer 

•  Canopy  damaged 

•  Canopy  distortion 

This  allows  the  trainee  to  become  familiar  with  the  recognition  of  good  and  bad  canopies  and  the 
procedures  that  should  be  followed  in  the  event  of  a  malfunction. 

The  trainee’s  control  inputs  and  flight  path  are  recorded  for  review  after  landing,  giving  the  instructor  the 
option  to  explain  any  incorrect  actions  taken. 

Initial  testing  of  the  simulator  by  experienced  and  student  parachutists  has  shown  that  it  offers  potential 
benefits  for  parachute  training. 

It  is  hoped  that  the  future  work  will: 

•  Extend  the  modelling  to  allow  for  different  canopy  types. 

•  Improve  the  visual  sub-system  allowing  for  greater  depth  perception,  weather  effects  and  day / 
night  capability. 

•  The  ability  to  use  real-world  terrain. 

•  Improve  the  input  device  mechanism  so  as  to  have  greater  range  of  movement. 

•  Incorporate  the  simulator  into  a  network  to  give  the  ability  for  team  training. 

4.5.3  Recently  Published  Literature 

Smith,  J.  and  Steel,  J.,  “The  Use  of  Virtual  Simulation  for  Dismounted  Infantry  Training”,  NATO 
Modelling  &  Simulation  Conference,  Royal  Military  College  of  Science,  Shrivenham  (RMCS),  England, 
October  2000. 

Sastry,  V.,  Trott,  E.,  and  Steel,  J.,  “A  Virtual  Environment  for  Naval  Flight  Deck  Operations  Training”, 
NATO  Research  &  Technology  Organisation  Meeting  Proceedings:  MP-058,  March  2001. 

Hollis,  Dr.  J.E.L.,  Meinertzhagen,  Maj.  R.D.,  and  Lai,  Maj.  N.,  “Application  of  Virtual  Reality 
Technologies  to  Communications” ,  IEE  Conf.  Pub.,  1997,  IPA97. 

Brent,  Maj.  P.,  ‘Networked  Virtual  Reality  for  Dismounted  Infantry,  ’  RMCS  July  1998. 

Ourston,  D.  and  Reece,  D.,  ‘Issues  Involved  with  Integrating  Live  and  Artificial  Virtual  Individual 
Combatants  ’,  SAIC  Stricom. 

Hutchinson,  Maj.  N.,  ‘The  Simulation  of  Dismounted  Infantry’,  RMCS,  2000. 

Clarkson,  Maj.  M.,  ‘Low  Cost,  3D,  Interactive  Networked  Training’  RMCS  1998. 

Jones,  Maj.  I.,  ‘Desktop  Dismounted  Infantry  Trainer  A  Proof  of  Concept  Demonstrator’,  RMCS  1999. 
Percival,  Maj.  A.E.,  ‘Distance  Estimation  in  Virtual  Environments  ’,  RMCS  2001. 

Bosher,  Maj.  P.  'Training  Transfer  in  Environmental  Awareness  from  Virtual  Environments’,  RMCS 
2001. 


RTO-TR-HFM-1 21 -Part-1 


6  - 13 


INSTITUTES  IN  THE  UK  UNDERTAKING  RESEARCH  INTO  AUGMENTED,  MIXED  AND 
VIRTUAL  ENVIRONMENTS  (AMVE)  FOR  MILITARY  APPLICATIONS  OF  VIRTUAL  REALITY 


ORGANIZATION 


4.6  University  of  Surrey 

4.6.1  Synthesising  Human  Motion  for  Virtual  Performance 

Recent  research  has  achieved  automatic  reconstruction  of  realistic  3D  models  from  surface  measurements 
of  static  objects  and  environments.  Currently  captured  object  models  are  represented  as  dense  unstructured 
polygonal  meshes.  Direct  application  of  unstructured  meshes  for  computer  generated  imagery  is 
prohibitively  expensive.  The  proposed  research  addresses  the  automatic  reconstruction  of  functional 
models  that  are  optimised  according  to  the  requirements  of  a  particular  application.  Functional 
requirements  include  realistic  and  efficient  representation  of  the  development  of  techniques  to  enable 
automatic  reconstruction  of  functional  models  of  real  objects  suitable  for  realistic  computer  generated 
imagery.  Applications  include  realistic  object  modelling  for  virtual  environment,  animation  and  Internet 
transmission.  In  particular  application  of  this  research  will  be  based  on  existing  expertise  in  capturing  3D 
models  of  complex  organic  objects  and  internal  environments. 

4.7  School  of  Computer  Science  Information  Technology 

4.7.1  The  Realisation  and  Utility  of  Persistence  in  Collaborative  Virtual  Environments 

The  design  and  authoring  of  a  collaborative  virtual  environment  is  normally  distinct  from  its  activation 
and  use.  For  example,  completed  virtual  worlds  are  made  available  as  fixed  definitions  (e.g.  world 
description  files)  which  are  used  repeatedly  to  create  distinct  instantiations  of  active  virtual  worlds. 
This  project  will  investigate  the  implementation  and  utility  of  collaborative  virtual  environments  for  which 
design,  creation,  publishing  and  use  are  not  differentiated  and  are  available  to  all  participants.  This  builds 
on  the  proposer’s  PhD  work  in  the  area  of  large-scale  collaborative  virtual  environments,  which  include 
the  implementation  and  ongoing  testing  of  the  MASSIVE-1  and  MASSIVE2  CVE  systems.  From  a 
distributed  systems’  perspective,  the  project  will  identify  and  employ  system  and  programming  facilities 
to  implement  persistent  virtual  worlds.  These  will  be  integrated  with  an  existing  virtual  reality  system  or 
tool  kit.  From  a  human  factors  perspective  the  project  will  make  one  or  more  such  worlds. 


5.0  UK  COMMERCIAL  COMPANIES 

5.1  Facit  -  A  Division  of  TWG  Ltd.13 

Facit  are  developers  of  Sense8  virtual  reality  authoring  software,  building  commercial  visualisation 
applications  for  a  wide  range  of  industries  including  retail,  education,  construction  and  engineering. 
Augmented  Reality  (AR)  is  a  growing  area  in  their  virtual  research.  Augmented  Reality  can  be  integrated 
into  a  number  of  different  industries  as  a  turnkey  solution.  Current  AR  users  include  Academics,  Medical, 
Entertainment,  Military  Training,  Engineering  Design,  Robotics  and  Telerobotics,  Manufacturing  and 
Consumer  design.  Facit  aim  to  bring  a  solution  to  industries  looking  at  the  implementation  of  Augmented 
Reality  into  their  current  research  projects.  They  have  had  success  in  supplying  AR  Head  Mounted 
Displays  and  also  the  solution  of  constructing  personalised  AR  unit  from  products  currently  available 
within  the  IT  market. 

In  addition  they  supply  simulators  for  the  defence  market  which  includes  large-scale  operations  planning 
and  mission  rehearsal  to  part-task  training,  using  a  blend  of  VR  and  multimedia.  Multiple  students  can 
familiarise  themselves  simultaneously  with  complex  systems,  vessel  or  craft  layout,  maintenance  tasks, 
fault-finding,  repair  and  refit. 


13  http://www.facit-vr.co.uk/defence_simulation.litm 
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Augmented  Reality  for  Familiarisation  of  Vessel  Layout. 


5.2  Advanced  Interactive  Solutions  Ltd.14 

AIS  work  in  partnership  with  large  system  integrators  and  strategic  partners  to  deliver  comprehensive 
training  solutions.  AIS  integration  of  simulation  technology  and  readiness  certification  through  advanced 
information  systems  is  ideally  suited  for  readiness  assurance  for  military  forces.  In  April  2003  AIS 
announced  the  sale  of  a  new  computerised  training  simulator  to  the  UK  MoD.  The  UK  MOD  has 
purchased  an  AIS  P3000  Live  Fire  PRISim™  System.  The  system  will  be  used  for  close-quarter  battle 
training  and  counter-terrorist  training  and  provides  highly  realistic  use-of-force  training  that  develops  the 
skills  required  for  personnel  armed  with  both  lethal  and  non-lethal  weapons.  Flexible  deployment  options 
allow  training  to  be  delivered  at  fixed  facilities  or  at  any  off-site  location. 

The  PRISim™  platform  produces  exceptionally  realistic  video-based  environments  for  all  aspects  of 
firearms  handling  including  marksmanship,  decision-making,  and  tactical  strategies  and  uses  broadcast- 
quality  DVD  technology. 


14 


http://www.ais-sim.com/casestudies_military.htm 
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Firearms  Training  Handler. 

(http://www.ais-sim.com/news_press_releases_060703.htm) 


Sim  Room. 


6  - 16 


RTO-TR-HFM-1 21 -Part-1 


INSTITUTES  IN  THE  UK  UNDERTAKING  RESEARCH  INTO  AUGMENTED,  MIXED  AND 
VIRTUAL  ENVIRONMENTS  (AMVE)  FOR  MILITARY  APPLICATIONS  OF  VIRTUAL  REALITY 


5.3  Inition  Ltd.15 

Inition  deliver  bespoke  solutions  to  clients,  having  years  of  experience  working  with  the  following 
technologies: 

•  Virtual  Reality 

•  Stereo  Visualisation 

•  Interactive  Art  &  Visuals 

•  Systems  Integration 

•  Web  3D,  Bespoke  Application  Development 

5.4  Maelstrom  Virtual  Productions  Ltd.16 

Virtual  Reality  training  and  simulation  environments  have  been  built  in  the  fields  of  medicine,  energy, 
architecture,  and  Military. 

5.5  Virtual  Reality  Centre  at  Teesside  Ltd.17 

The  company  is  involved  in  image  generation,  visualisation  and  training  simulations.  It  creates  PC-based 
and  internet  off-line  visualisation  and  training  packages  for  various  industries.  Their  products  include: 

The  Hemispherium®  -  a  6m  dome  using  7  projectors  to  give  a  full  180°  field  of  view  and  driven  by  a 
3  pipe  Silicon  Graphics  IR2. 


The  Hemisperium®. 

The  Auditorium  -  a  7m  wide  Cylindrical  Display  using  3  Projectors  to  give  a  140°  x  45°  Field  of  View. 


15  http://www.inition.co.uk/ 

16  http://www.maelstrom.com/military.htm 

17 

http  ://www.vr-centre  .com/ 
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The  Auditorium. 


5.6  cueSIM  (ex-Motionbase) 

Originally  developed  by  Motionbase  pic  at  Bristol,  production  is  now  by  cueSIM  at  Bedford.  QinetiQ 
acquired  Motionbase  in  2002  and  formed  cueSIM.  The  company  has  access  to  the  simulation  expertise  of 
its  owner  organisation,  the  UK  company  QinetiQ  pic  that  deals  in  R&D  for  the  UK  Ministry  of  Defence. 
This  may  be  particularly  important  in  aspects  such  as  multi-player  networking  and  the  use  of  Computer 
Generated  Forces  (CGF).  Their  products  include: 

•  VANguard  multi  player  simulators:  VAN  stands  for  Virtual  Air  Network  and  this  range  of  devices 
is  designed  to  be  coupled  together  for  mission  training  exercises.  A  modular  design  is  employed 
and  types  from  desktop  to  motion-based  Full  Flight  Simulators  are  available.  Facilities  include 
DIS,  HLA  and  CGF. 

•  RTAVS  simulation:  This  stands  for  Real  Time  All  Vehicle  Simulation  and  includes  DIS/FILA 
compliance  and  compatibility  with  many  COTS  components.  RTAVS  is  essentially  a  software 
architecture  that  allows  hardware  to  be  rapidly  implemented. 

•  Motion  platform:  CueSIM  Maxcue  6-DoF  electric. 

•  Explorer  dynamic  flight  simulator:  This  includes  a  compact  flight  simulator  cockpit  shell  mounted 
on  an  electric  motion  platform.  The  display  system  comprises  three  of  the  company’s  collimated 
monitors. 


6  - 18 
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5.7  Pennant  Training  Systems  Limited  (PTSL)18 

5.7.1  Virtual  Aircraft  Training  System  (VATS) 

This  is  a  computer-based  classroom  training  system  for  aircrew  and  maintainers. 

Features  include  interactive  training,  maintenance  and  diagnostics,  aircraft  systems  emulation  with  faults, 
test  equipment  and  LRU  change. 


The  Virtual  Aircraft  Training  System. 


18  http://www.pemiantplc.co.uk 
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5.7.2  Synthetic  Environment  Procedural  Trainer  (SEPT) 

The  Royal  Air  Force  has  adopted  the  very  latest  development  in  Virtual  Reality  to  enable  training  of 
recruits  in  marshalling  and  ground  handling  of  aircraft.  The  trainer,  known  as  the  Synthetic  Environment 
Procedural  Trainer  (SEPT)  is  in  service  at  RAF  Cosford  where,  after  classroom  instruction  on  the  required 
skills  and  safety  procedures,  students  consolidate  their  learning  through  practical  experience  on  the  SEPT. 
This  is  an  ideal  preparation  ground  for  the  real  situation  where  the  marshaller  must  give  clear  and  positive 
direction  to  the  aircraft  pilot. 

SEPT  is  a  complete  procedural  trainer  providing  high  fidelity,  real-time,  computer  generated  images  of  an 
operational  airfield  with  realistic  visual  and  aural  representations  of  operational  aircraft,  general  airfield 
activities,  buildings  and  vehicles.  Central  to  the  SEPT’s  capability  is  the  high-fidelity  visual  system  with 
its  wide  field  of  view  (FOV).  The  aircraft  visual  is  integrated  into  the  SEPT  computing  system  that 
contains  an  aural-cueing  database,  synthetic  airfield  database  and  Instructor  operating  facilities. 


The  Ground  Marshalling  Training  Simulator. 

The  visual  database  is  projected  onto  a  wide-angle  screen  in  front  of  the  trainee  who  is  then  able  to 
conduct  the  Marshalling  and  Ground  Handling  of  Aircraft,  in  an  immersive  environment.  This  enables  the 
trainee  to  progress  from  a  theoretical  base  into  the  practical  environment  and  instills  confidence  in 
undertaking  the  task  for  real. 

5.8  Virtalis  -  Naval  Virtual  Reality  Close-Range  Gunnery  Trainer  (CRGT)19 

Virtalis  employs  an  experienced  team  of  software,  hardware,  electronics,  human  factors  and  design 
specialists.  They  form  partnerships  whatever  type  of  work  is  needed,  be  it  a  short  commercial  feasibility 
study,  an  in-depth  technology  review,  a  concept  demonstrator  or  a  full-blown  visualisation  system. 


19  http://www.vrweb.com/ 
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